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Abstract. We derive a formula for the first variation of horizontal perimeter 
measure for C 2 hypersurfaces of completely general sub-Riemannian manifolds, 
allowing for the existence of characteristic points. For C 2 hypersurfaces in 
vertically rigid sub-Riemannian manifolds we also produce a second variation 
formula for variations supported away from the characteristic locus. This 
variation formula is used to show the bubble sets in H 2 are stable under volume 
preserving variations. 



1. Introduction 

Optimization problems lie at the heart of many pure and applied problems, two of 
which, the minimal and isoperimetric surface problems, have played a central role in 
mathematical development over the last century. In the last decade, there has been 
increasing interest in these problems in the setting of sub-Riemannian spaces. This 
interest is driven in part by applications of sub-Riemannian geometry to optimal 
control as well as to more novel applications such as the recent sub-Riemannian 
model of the primary visual cortex [8, 25, 34, 35]. We are also motivated by a deep 
conjecture of Pansu [32] concerning the isoperimetric profile in the sub-Riemannian 
Heisenberg group, which has seen a great deal of recent activity with many partial 
results [12,27,31,37] (see also [5] for an overview of this problem). 

One of the basic approaches to such optimization problems uses the tools of the 
calculus of variations to determine the geometric and analytical properties of their 
solutions. Recent investigations of minimal and isoperimetric surfaces have focused 
on this approach with many authors deriving first and second variation formulae. In 
[1,3, 11, 12,33,36,37], the various authors compute first variation formulae for C 2 
smooth noncharacteristic surfaces in the Heisenberg group. We note that some of 
these authors restrict their attention to certain types of graphs (Euclidean graphs: 
[33,37], intrinsic graphs [1]), [11,12] deals with a level set formulation and [3] 
provides a completely general nonparametric first variation formula. In [6] the 
authors compute a first variation formula for C 2 noncharacteristic graphs in any 
three dimensional pseudo-hermitian space (including, of course, the Heisenberg 
group). In [30,38] the authors independently provide a first variation formula for 
C 2 noncharacteristic surfaces in general Carnot groups. In [9,39], the authors 
provide a first variation formula for C 2 surfaces in Martinet-type space and (2,3) 
contact manifolds (respectively). 
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In [24] , we compute the first variation formula for C 2 noncharacteristic surfaces 
in all so-called vertically rigid spaces. To formalize this, we recall some of the basic 
definitions. 

Definition 1.1. A sub-Riemannian (or Carnot-Caratheodory) manifold is 

a triple (M, Vo, (•, •)) consisting of a smooth manifold M of dimension n + 1 = 
k + I + 1, a smooth k + 1- dimensional distribution Vo C TM and a smooth inner 
product on Vq. This structure is endowed with a metric structure given by 



where is the space of all absolutely continuous paths whose derivatives, when 
they are defined, lie in Vo- 

Definition 1.2. A vertical complement to sub-Riemannian structure is 

• a smooth complement V to Vo in TM 

• a smooth frame T\ , . . . 7] for V 

• a Riemannian metric g such that V and Vo are orthogonal, g agrees with 
(•, •) on Vo and T\, . . . Ti are orthonormal. 

A vertical complement is rigid if in addition 

• there exists a partition of {1, ■ ■ ■ ,1} into equivalence classes such that for all 
sections X G T{V ), g{[X,T a ],T f3 ) = 0i/a~^. 

A sub-Riemannian space with a rigid vertically complement is called a vertically 
rigid (VR) space. 

We note that all of the cases of the first variation formula described above are 
vertically rigid and, hence, our first variation formula generalizes all of these. In 
fact (see Theorem 5.5), a slight modification of the argument from [24] allows us to 
compute a first variation formula in any sub-Riemannian manifold: 

Theorem A. Suppose S is aC 2 noncharacteristic hypersurface in a sub-Riemannian 
manifold M and F is a C 1;2 variation of S with horizontal variation function po- 



Here, Po denotes the sub-Riemannian perimeter measure and H denotes the 
horizontal mean curvature (see Section 3 for a precise definition) . As a consequence, 
we have: 

Theorem B. A necessary and sufficient condition for a C 2 hypersurface E to be a 
noncharacteristic critical point for the horizontal perimeter measure in the category 
of C 1 hypersurfaces with fixed boundary in a sub-Riemannian manifold is 



If the vertical structure is rigid, the second term drops out and the equation becomes 




Then 




divu = H-^2([v,Tp],Tp) =0. 



div v = H = 0. 
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Recent work of Cheng-Hwang- Yang [7] and Ritore-Rosales [36] have shown how 
to extend the first variation formula to allow for variations over the characteristic 
locus in the Heisenberg group. Our first main result of this paper is to prove a sim- 
ilar extension of the previous Theorems to include variations over the characteristic 
locus. 

Theorem C. Let E be a C 2 hyper surface in a sub-Riemannian space M with 
characteristic locus S(M). Further suppose that the Riemannian curvature tensor 
of E is bounded and that the horizontal mean curvature of E, H, is in 
Suppose F is a compactly supported C 1 ' 2 variation of E with F C 2 and variation 
function p. Then 



Here, N is the Riemannian normal to E, v is the unit horizontal normal and v T 
is the component of v tangential to E. 

In the previous theorem, the family $1$ C E, 5 > satisfies the following condi- 



• The portion of the boundary dfls in the interior of S is piecewise C 2 . 

• C(E) C fl 5 for all S > 

• p n (n s ) -> as (5 -> 0. 

• If we let iVo be the projection of the Riemannian normal to V, then as 
^0»/sn 4 \No\dV d n 5 ^0. 

Here p n denotes the n-dimensional Riemannian spherical Hausdorff measure. 

We note that due to work of [2, 28] on the size of the characteristic locus, as well 
as computation in the appendix, we can show that such a family of sets always 
exists. 

In keeping with historical terminology, we call critical points of the perimeter 
variation minimal surfaces. 

As an application of the general first variation formula, we prove a version of the 
Minkowski formula in this setting (see Theorem 6.4 and Corollary 6.5): 

Theorem D. Suppose Q is a compact C 2 domain with E = d£l that is a critical 
point for perimeter measure with volume constraint. Then 



We note that this formula was shown in groups of Heisenberg type in [12] and 
in the Heisenberg group in [36] . 

A number of authors [11,30,38] have also computed second variation formulae 
in the setting of Carnot groups as a tool in the investigation of stable minimal 
surfaces. As has been shown recently [1, 13-15], stability plays a crucial role in the 
study of minimal surfaces in the Heisenberg group. Specifically these papers study 
analogues of the sub-Riemannian Bernstein problem and show that without the 
imposition of stability on critical points of the first variation of perimeter, there is 
no Bcrnstcin-typc rigidity. On the other hand, there is rigidity in the presence of the 
stability condition. The most general of these results is an analogue of Riemannian 
results of Fischer-Colbcric/Schoen [18] and Do Carmo/Peng [17]: 




tions: 



(Q-l)Po(S) = QHVol{Q). 
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Theorem 1 ([15]). The only stable C 2 complete embeded noncharacteristic minimal 
surfaces without boundary in the first Heisenberg group are the vertical planes. 

To facilitate further study of stable minimal surface, we derive a second variation 
formula for C 2 noncharacteristic surfaces in vertically rigid spaces (Theorem 7.4): 

Theorem E. Suppose M is a vertically rigid subRiemannian manifold and F is 
a noncharacteristic C°° :3 variation o/£\C(£) with compactly supported horizontal 
variation function. Then 



dt 2 



P (X t ) = jf [(d t po)HA} lt=0 + jf |V°' s p f A 

+ J p 2 [ - ffic v (^, v) - Tr(n 2 ) + H 2 

(1) -{Toiiu^Tp^Torie^T^v) 

- ( Tor(e 3 , Tp),N v ) ( Tor{v, e 3 ), T fj ) 

- [ej{Toriv,ej),N v ) -2(Tor(v,e 3 ),N v )(V e 

- (Tor{v,ej),Tp){ejap) - (Tor(v,e 3 ), N v ) 2 A 



I ^3 5 ^"i 



Again, v is the unit horizontal normal to S, {ej} form an orthonormal basis for the 
horizontal tangent space to X, and the {Ti} is the family of vertical vectors fields 
from the definition of a vertical complement. IIo denotes the horizontal second 
fundamental form. The connection V is adapted to the vertical structure and all 
torsion terms are associated to V. 

We note that this second variation formula, when restricted to the special case 
of the Heisenberg group matches with others in the literature [7, 11, 12, 36]. As a 
last application, we show that the conjectural isoperimetric profile in H 2 is indeed 
a stable constant mean curvature surface (Section 9). 



2. Notation and conventions 

To improve economy with the intensive computations throughout this paper, we 
shall following the following notations and conventions: 

(A) Unless explicitly stated otherwise, roman indices will run from 1 . . . k, barred 
roman indices from ... k and greek indices from 1 . . . I. 

(B) ljU' denotes the ordered wedge product of all possible (by index conven- 
tions) 1-forms lu 1 with the jth form omitted, e.g 

w (2) = uj 1 A uj 3 A • • • A u k 
uj 2 = uj° A uj 1 A uj 3 A • • • A uj k . 

We shall also use with i < j to denote the ordered wedge product 

with both ith and jth terms missing and extend to all indices by setting 
W W) = - w 0'>*). 

(C) (Summation Convention) Whenever the same index appears twice in a term 
obeying the above conventions, we shall assume that there is an implicit sum 
over all possible values. 
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3. SUBRlEMANNIAN MANIFOLDS AND VERTICAL STRUCTURES 

We begin with our basic definitions: 

Definition 3.1. A sub-Riemannian (or Carnot-Caratheodory) manifold is 

a triple (M, Vo, (•, •)) consisting of a smooth manifold M of dimension n + 1 = 
k + I + 1, a smooth k + 1- dimensional distribution Vq C TM and a smooth inner 
product on Vo- This structure is endowed with a metric structure given by 



where is the space of all absolutely continuous paths whose derivatives, when 
they are defined, lie in Vq. 

We shall also make the standing assumption that M is oriented. 

Definition 3.2. A vertical complement to sub-Riemannian structure is 

• a smooth complement V to Vo in TM 

• a smooth frame 7\ , . . . TJ for V 

• a Riemannian metric g such that V and V a are orthogonal, g agrees with 
(-, ■) on V and 7i, . . . TJ are orthonormal. 

A vertical complement is rigid if in addition 

• there exists a partition of {1, ■ ■ ■ ,1} into equivalence classes such that for all 
sections X £ T(Vq), g([X,T a ],Tp) = if a ~ (3. 

A sub-Riemannian space with a rigid vertically complement is called a vertically 
rigid (VR) space. 

For convenience of reference, if M is a VR space we decompose 



where B is the set of equivalence classes in {1, . . .1} and V& = span{T,g: (3 G b}. 
We shall also use Lb = \b\. If the vertical structure is not rigid we use the same 
notation with the understanding that the equivalence relation is simply equality, 
i.e. a <~ [3 if and only if a = [3. 

Many of the essential computational tools of Riemannian geometry can be gen- 
eralized or restricted to include subRiemannian geometries with vertical structures. 

Definition 3.3. A connection V is adapted to a subRiemannian geometry with 
vertical structure if 

• V is compatible with g 

• VT p = for = 1... I 

• Tor(X, Y)\ p e V\ p for all sections X, Y of Vq 

These connections were first defined in [24] for VR manifolds but the rigidity 
assumption is not necessary. The following properties of adapted connections were 
also proved in [24]: 

Lemma 3.4. 

• Every vertical structure admits an adapted connection. 

• For X,Y sections ofVo, VxY is depends solely on the vertical complement 
V and not the Riemannian extension g. 
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• IfVis adapted to a rigid vertical structure and X is a section of Vb then 
(Tor(T a ,X),Tp) = 0, ifa~0. 

Remark 3.5. It is the vanishing of these torsion terms that makes VR structures 
much easier to work with than general subRiemannian manifolds. 

For a C 1 hypersurface E in a subRiemannian manifold, we define the character- 
istic set of E to be 

C(E) = {peE: (Vb)| p CT p E}. 

For an oriented C 1 hypersurface E C M we define N to be the unit Riemannian 
normal to E with respect to g and No as the orthogonal projection of N to Vq. 
Away from the characteristic set, we define the unit horizontal normal 

V |JVo| 

Definition 3.6. The horizontal -perimeter measure of E is defined to be 

Po(E) = J \N \dVj: 

where dVs = N_idV g . 

For noncharacteristic surfaces, Pq has the alternative descriptions 

Po(E) = J vjdVv = sup |y XjdV^ : X e T(V ), \X\ = l| 

There are several natural questions associated to this perimeter measure. 

Question 1. Among hypcrsurfaces with the same boundary, which minimizes the 
horizontal perimeter measure? Can such surfaces be characterized as solutions to 
a PDE? 

Question 2. Among domains of the same volume, which has boundary minimizing 
horizontal perimeter measure? 

These problems are studied using variational techniques which may yield critical 
points rather than true minima. Thus there is another natural question: 

Question 3. Of the critical points of horizontal perimeter measure, which are 
stable, i.e. -^\ t _ P (Y^ t ) > for any variation of E. 

Under the assumptions of rigidity, no characteristic points and C 2 regularity, 
Question 1 and Question 2 were answered in [24] in terms of the horizontal mean 
curvature. 

Suppose eo, • • • forms a (local) orthonormal frame for Vb such that on E\C(E), 
eo is the unit horizontal normal to E. Then away from C(E), the horizontal second 
fundamental form for E is defined by 

/(V ei e ,ei) ... (V ei e ,e fe }\ 

//■■ : : : 

V(Ve fe e ,ei) ... (V ek e ,e k )J 

The horizontal mean curvature is defined by 
(2) H = tracc(iZo). 
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We remark that the connection used in these definitions can be cither the Levi- 
Cevita connection for the Ricmannian metric or any connection adapted to the 
vertical structure. 

In a VR manifold, C 2 minimizcrs of Pq with fixed boundary constraint were 
shown in [24] to satisfy H = away from characteristic points. Likewise C 2 mini- 
mizers subject to the volume constraint satisfied the condition that the horizontal 
mean curvature was locally constant away from the characteristic set. 

We finish this section by making some remarks on the nature of the equation 
H = c off the characteristic set. Since any adapted connection is metric compatible, 
we obtain the following result about the ambient divergence on M from standard 
results in Ricmannian geometry (see [26], appendix 6): 

div Z = tracc(VZ + Tor(Z, •)) 

Thus if Xj denotes a local horizontal orthonormal frame and Z is a horizontal vector 
field 

div Z = ( V Xi Z , Xi ) + < Tor(Z, 7» , 7> ) 
= {Vx- i Z,X- i )-{[Z,T /3 ]),T ). 

Applying this to v yields 

dxvv = H-{[v,T p ]),Tp). 
In the rigid case, the second term drops out and H naturally takes the form 

H = div v. 

Elsewhere in the literature, for example [36], the hypersurface divergence has 
been used instead of the ambient divergence. For completeness we shall now show 
that the two approaches are equivalent. In particular, our variation formula agrees 
with that derived by Rosales and Ritore for the special case of the first Heisenberg 
group, . 

Definition 3.7. For vector fields Z G T(TM) we define the surface divergence of 
Zatpe^by 

(div^Z)uj = i*£z<jJ 

at p, where u) — N_idV for some extension of N to a unit vector in a neighbourhood 
o/S. 

We note that it is easy to show that if Z = Z' along S then div^ = div^Z'. 

Lemma 3.8. Suppose S is a C 2 hypersurface in a VR manifold M and X\, . . . X n 
is an orthonormal frame for T,. Then for any horizontal vector field Z 

n 

diwZ = ^2(V L , X] Z , uXj) 
for any connection V adapted to the VR structure of M. 

Proof: We adapt the same argument from [26] as follows: since V is metric com- 
patible we see 

n 

Vw = V (bj 1 A • • • A w"- 1 ) =^w°Acj 3 A w«) 
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Thus i*Vw = 0. Therefore if we identify Xj with L*Xj, 
0=[f (Vz-jC z )}u;(X 1 ,...X n ) 
= -l* {C z u)(X u ...X n )+J2 ■ • • (Vz - C z )Xi, . . . X n ) 

n 

= -(div s ZV(X 1; . . . ,X n ) + "(Xi, ■■■Vx i Z + Tor(Z, A,), ...X n ) 

= ^-div E Z + J2( V Xj Z, Xj^j w(X u ...X n ) 

where the last line follows from the fact that ( Tor(Z, X,) , Xj ) =0 whenever Z is 
horizontal. This last observation fails in the non-rigid general case. 

■ 

Corollary 3.9. Away from the characteristic set C(S) we have 

div v — divsv. 

Proof: Away from C(S) we can choose an orthonormal frame for £ of the form 
{e-j} k jZ[ , ep — spv + cpTp where the Bj are horizontal and at least one cp ^ 0. Then 

div s z^ = ( \7 g/3 v , ep ) + ( V ej v , ej ) 
= ( Vg^v, +div j/ 
= div v. 



4. Bundles and Variations 

To describe the variational properties of the horizontal perimeter measure, we 
shall define a variety of bundles over M. 

First, we shall denote by S(M) the contact manifold of normalized hyper- 
surface elements, i.e it: S(M) — > M is the unit tangent bundle over M viewed 
as a bundle of Riemannian unit normals. We define the 1-form O on S(M) by 

®\(p,e)( X ) = (**X,E) P . 
An immersion i of an n-dimensional manifold into S(M) is said to be transverse 
if 7T o l is an immersion and i*9 = 0. 

Definition 4.1. The function N : S(M) — > Vb(M) is defined by 

N (p,E) = ((ir*E) )\ p 

Here, we use the convention that if W is a vector field on M then (W)q is its 
projection to a vector field in Vo . 

The characteristic slice, Cs of S(M) is the zero level set of No- 

There is a natural projection 7Tjf from the Riemannian frame bundle T{M) to 
S(M) given by 

7rjr : 0, E ,... E n ) i > (p, S ). 
We note that if E°, . . . E n denote the tautological forms on T{M), (i.e. at the point 
(p, So, ■ • • S„), E*(X) = (ir*X, , j = 0, . . . ,n) then 

6 = o-*S° 
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for any section a of 7Tjf. 

Definition 4.2. A differential form ip on S(M) is semibasic if 

wherever ir*X = 0. Thus depends only the the projection to M and the choice of 

Eq. 

For example, it is clear that 6 is a semibasic 1-form. 

The bundle of graded orthonormal frames is the subbundle Q (M) C T{M) 
such that 

(E ... E n ) = (e • • • e fe , t u , . . . , ti Li t\ B \ x , t\B\ L[B[ ) 

where the e/s are all horizontal and t^,.. . ,tb L span V&. The reduced structure 
group of the bundle is then 0(k + 1) x H beB 0(L b ). 

For computational purposes it is often easiest to further restrict and insist that 
vertical frames be exactly those specified in the definition of the vertical structure. 
If such a vertical structure has been chosen, we shall note this further restriction 
by GT V {M). 

Unfortunately, these graded bundles do not encode enough information to de- 
scribe the geometry of hypersurfaces of M. To compensate for this we also introduce 
the augmented bundles 

The additional elements will be used to keep track of the dependence of the hyper- 
surface normal directions on the vertical vector fields. 

If an explicit vertical structure has been fixed, there is an alternative presentation 
of S(M)\Cs that will prove computationally simpler to work with for noncharac- 
teristic variations. 

We define the contact manifold of horizontally normalized hypersurface 
elements to be 

5 (M) = {(p, Z) G TM :\Z\ = 1,Z€ (V Q ) P } x R l . 
There is a bundle isomorphism S(M)\Cs = So(M) given by 
(p, E) i ^ (p, l^ol" 1 ^, IJVbl" 1 ^,^)) 

with inverse 

(P, eo,ap)^ (P, , , .„ (e + apTp)) . 

We shall identify <So(M) with S{M)\Cs using this bundle isomorphism. 
We define a 1-form 9 on Sq(M) by 

9(X) = faX^o + apTf)) 

and note that on So(M) we have 

|JV | = ] and 9 = \No\0. 

There is a natural projection from ng : QT(M) — ► Sq{M) such that if 
r] 1 . . .rf are the tautological 1-forms for QF{M) then 

9 = <T*{u)° + aprf) 
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for any section a of irg. Since we shall frequently be computing on the frame 
bundles, we shall often implictly identify lu° + aprf with 9. 

For the remainder of this section, we shall suppose that £ is an oriented, im- 
mersed C 2 hypersurface of M realized as the image of the C 2 immersion 

t: S ^ M 

for some smooth oriented manifold (possibly with boundary) E. 
Definition 4.3. A variation of X is a map 

F: Ex (-e,e) -» M 

such that 

• Each F t = F(-,t) is an immersion of E into M. 

• F = l. 

The lifted variation F : E x (— e, e) — > 5(M) is t/ie map defined by 

F&t) = (F{£,t),N\ Fm ) 

where N is the (local) Riemannian unit normal vector to the immersed surface 
F t (E) such that the pullback of N jdV matches the fixed orientation ofE. 

The variation function of F is p = p(-,0) where F*Q — pdt. The variation is 
said to be compactly supported ifp(-,t) has compact support for all t. 

Definition 4.4. When the lifted variation F maps into the complement of the 
characteristic slice Cs, we shall refer to the variation as noncharacteristic. The 
horizontal variation function for F is then defined by po = /5o(-, 0), F*9 = podt. 

Remark 4.5. Since Cs is closed, z/S has no characteristic points then, shrinking 
e if necessary, any variation F will map into So(M). The relationship between the 
variational functions is just 

p=\N o F\p . 

So far, we have not put any regularity conditions on our variations. However, 
we shall need precise descriptions of regularity to make our theory optimal. 

Definition 4.6. The classes ofC v 'i maps from K" x M t to R for i, j > are defined 
inductively by 

• C 0;0 = C°, i.e. continuous maps. 

• Ft C n+1 '° if and only if F, G C i;0 for all m = 1 . . . n. 

• F e C°-'j +1 if and only if F, % G C '^ . 

• F € C i+1 'i +1 if and only if F e C^ +1 n C l+1 ^ ,^ G C l+1 ^ and G 

f or all m = 1 . . . n. 

Thus a map is C 1 '^ if up to i continuous spatial (x) derivatives and j temporal (t) 
continuous derivatives can be taken in any order. 

Using coordinate charts, this definition extends naturally to define C 1 '^ maps 

for smooth manifolds E and M . 
Remark 4.7. We note in passing that 

i+j=m 
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The following approximation result will be useful later 

Lemma 4.8. Given a C 1 '^ map F: SxI^M that is constant outside K xR for 
some compact set K C S, there exists a sequence F m of C°°'^ maps such that 

(1) F m converges to F in C V ' J . 

(2) IfF(-,t ) G C' p then F m (-,t ) converges to F(-,t ) in C p . 

This lemma is essentially a version of standard approximation theorems adapted 
to allow parameters. The reader is referred to [21] pp. 41-55. for a proof that C r 
maps between smooth manifolds can be approximated by smooth maps. Theorem 
2.3 in [21] can easily be adapted to give an approximation of C 1 '- 7 maps from HxE — > 
R m by C 00 ' 3 maps, with the observation that the mollification process should only 
be in the S coordinates. Everything else goes through virtually unchanged. 

Lemma 4.9. If F is a C v ^ variation i, j > 1, then F is a C 4 ~ lj map. 

Proof: The tangent space to the immersed surface F t (£) is locally spanned by 
the vector fields F^d^m. Therefore the Gram-Schmidt algorithm followed by a 
horizontal projection and rescaling, expresses the unit horizontal normal to -Ft(S) 
v as a smooth combination of these spanning vector fields. Thus v can be viewed 
as a C* -1 '- 7 ' function. 

■ 

We now list a few basic regularity properties 

Lemma 4.10. If F is a C v ^ variation and tp is a smooth semibasic differential 
form on S(M) then F*ip is a C" -1 '- 7 ' -1 form onE x (— e, e). 

Proof: The real issue here is that as a map into S(M), F only has C* -1 '- 7 ' regularity. 
However, F*ip depends tensorially on the projected input F*-^, F*-^, which are 

C* ;J ' -1 , C* -1 '- 7 ' vector fields respectively, and its position F(x,t) which is also at 
least C 1 ' 1 ^. 

m 

In particular, the lemma implies that we can make sense of the pullback of 
semibasic forms by C 1 ' 2 variations despite the fact that the lifted variations are 
only continuous maps into S(M). 

Corollary 4.11. For a C v ^ variation F and smooth semibasic form ip, the form 
has C % ' 3 regularity. 

Proof: The proof is identical to the previous lemma except that we no longer need 
dependence on F* ^ . 

■ 

Corollary 4.12. For a C n ' 2 variation F, i > 1, the variation function is C* -1 . 

Furthermore, it will be of interest to note that, locally at least, every function 
on S can be realized as a variation function. 

Lemma 4.13. For every point p = t(£) € S there exists a neighborhood £ e U C S 
such that every function p, j — 1,2, on U is the restriction of the variation 
function for a C r '°° variation of S. 

This is shown using standard arguments with Pfaff coordinates (see [4], p. 16). 
The restriction j = 1, 2 is due to the fact that E is only assumed to be C 2 . 
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5. Horizontal Perimeter Measure and the First Variation 

Throughout this section we suppose M is a subRicmannian manifold with a 
chosen vertical structure. 

Recall that on n: QT(M) ^Mwe have the tautological 1-forms cj j and rf by 

w j (X) = (ir*X, ej ), j = 0...k 
rf{X) = (Tr*X,t j ), j = l...n-k 

When computing locally with a noncharacteristic variation, we shall use the pull- 
backs of the forms (aoF)*^ 1 ,. . . , (cro F)*uj k , (aoF)*^ 1 . . . (<ro F)*r] n ~ k together 
with dt as a frame on E x (— e, e). Here a is any section of ttq. We shall drop the 
(<7 o F)*'s when referring to this framing. 

Remark 5.1. We shall refer to the pullback of A to So(M) also as A and to the 
pullback to E of A by F t as A t . 

Lemma 5.2. //£ is a noncharacteristic hypersurface of M and f is a tranvserse 
immersion £ ^> So(M) then 

PoV) = J J* A 
Proof: This is obvious from the definitions. 

■ 

Most computations will be undertaken on the frame bundles. In addition to the 
tautological forms, we also have the bundle structural equations for the connection 
(see [24]) 

dJ — lo™ A iA + T> 

(3) 

drf = r] a Ar?f + f . 

When working on QTv.o (as opposed to QT a ) the vertical torsion pieces take the 
form 

= drf. 

As was noted in [24], if the vertical structure is rigid then t@(X, tp) = 0. 
Definition 5.3. We define the 1-form ^ on QTq by 

Lemma 5.4. If u is any section of ng then 

dA = 9Ao-*y + a* ((-l) fc+/3 - V A f f} A r/^) . 

Proof: This was essentially proved in [24] section 4, but without the assumption 
of rigidity the torsion term cannot be dropped. 

■ 

The importance of "J/ lies in Lemma 5.4 and the fact that for a noncharacteristic 
variation of E, 

(ctoFq)** = HA . 
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Theorem 5.5. Suppose E is a C 2 noncharacteristic hyper surf ace in M and F is 
a C 1 ' 2 variation of E with horizontal variation function po- Then 



d_ 

It 



Po(E t ) 



4=0 



/ s Po (h-J2([u,T },T ) ] A. 



Proof: For the rigid case, this was proved in [24], section 4 . A very minor 
modification using Lemma 5.4 without dropping the torsion term gives the general 
case. For completeness, we sketch out the argument: first note 

d 



dt 



Po(E t ) 



C dt F*A 



Now A is a spatial form on <S (M), so we can only guarantee C 0;1 of F*A on 
5 x (— e, e). However, since the variation has C 1;2 regularity and E itself is a C 2 
hypersurface, at t = we can differentiate on QTyo to see 

d — r '---- 1 LC- ' 



dt 



Po(E t ) 



t=o 



L 



d(d t jF*A) 
d t jF*A 



+ 



t=o 



d f F*dA 



t=o 



t=o 



+ [ d t j{a of)* («Af+ ((-l)^- 1 ^* Af"A 77 (/3) )) 
(a o F )*y + (a o F)* (-( [v, Tp] , T p )u* A ry*) 



-L 



t=o 



Where we have used the fact that 

fV,7» = -<h7>],7>). 



Corollary 5.6. A necessary and sufficient condition for a C 2 hypersurface E to 
be a noncharacteristic critical point for the horizontal perimeter measure in the 
category of C 1 hypersurfaces with fixed boundary is 

div v = H -Y,([v,Tf,],Tf,) =0. 



If the vertical structure is rigid, the second term drops out and the equation becomes 

div v = H = 0. 

Remark 5.7. This is the first result of this nature for completely general subRie- 
mannian manifolds. The rigid case was shown in [24] . Prior results include numer- 
ous important cases: level sets in Carnot groups [12,30], for graphs in three dimen- 
sional strictly pseudoconvex pseudohermitian manifolds [6], Martinet-type spaces 
[9], for graphs in the Heisenberg group [20,33], for parametrized surfaces in the 
Heisenberg group [3], for intrinsic graphs in the Heisenberg groups [1], and for sur- 
faces in (2,3) contact manifolds [39]. 
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In the presence of characteristic points, the situation becomes more complicated. 
It is to this case that we now direct our attention. To avoid needless repetition, we 
shall make the following assumptions throughout this section. 

(A) £ is an oriented C 2 hypcrsurface with piecewise C 1 boundary in some n+ 1- 
dimensional VR manifold M. 

(B) The Riemannian unit normal to S will be denoted N. Off the characteristic 
set C(£), the unit horizontal normal v = jj^Nq. 

(C) E is the image of the immersion i: E — > M with E C K n . 

(D) F: E x (— e, e) — > M is a C 1;2 variation of £ with F a C 2 mapping. In 
particular this implies that F is a C 0;2 map. 

(E) The horizontal mean curvature of £, H € 

(F) The Riemannian curvature tensor of £ is bounded. 

Remark 5.8. For any transverse immersion f of £ info iSo(M), the N referred 
to above is equivalent to f*N n . Likewise we can pull N back to Ex (— e, e) and we 
shall not make any notational distinction between them. 

The necessary observation for studying variations for hypersurfaces with char- 
acteristic points is the following: 

Suppose F is a variation of £. Set C(E) = i~ 1 C(H) and note that C(E) is a 
closed subset of S. Furthermore by the results of the appendix, C(E) must have 
Hausdorff dimension < n — 1. Let U be any open subset of E containing C(E). 
By shrinking e if necessary, F induces a noncharacteristic variation Fh of T\i(U) 
as discussed in Section 4. Furthermore, if p is the variation function for F, then 
|7Vo|~V is the variation function for F H . In particular 

(d t ^F*@) ]t=0 dVj: = (d t ^F* H 9) lt=0 A. 

Before diving into the general first variation formula, we shall need some technical 
lemmas. 

Lemma 5.9. With the assumptions listed above, 

• | iV (- , 0)| is a Lipschitz function on Ex {t}. 

• |iVo(-,i)| is C 0;1 off C (lit) and has bounded distributional temporal deriva- 
tive on all of E. 



The one-sided derivative ^ 



|iV"o(t) | exists everywhere, is continuous off 

*=o+ 



C(E) and is bounded on E 

Proof: The first part follows from the fact that iVo(-, 0) is a C 1 map as Fq is C 2 . 
The remaining parts of the lemma are obvious properties of the absolute value of 
a C 1 function from R to 1. 

■ 

Lemma 5.10. Suppose U is an open set in E such that dll does not intersect 
C(E). Then for any vector field X on M, 

X jA| ac/ = | N 1 ( X , N du ) dV du - ( v , N du ) ( X , N ) dV du 

where Nqjj is the Riemannian unit normal inside E to dU . 

Proof: Note that off C(E), N = \N \v + \N \a f jT f:} for constants a p . The last 
piece can be rewritten as aT for some unit vector T orthogonal to Vq. Thus we can 



VARIATION OF PERIMETER MEASURE IN SUB-RIEMANNIAN GEOMETRY. 



15 



construct a vector field e along E\C(E) by e = av — \Nq\T. Then since we must 
have |7V | 2 + a 2 = 1, clearly v = \N \N + ae. 

Now X_iA = XjujdV, thus splitting v into pieces orthogonal and tangent to E 
we have 

XjA = |iVo|XjrfT/ s + [3XjejdV 
Pulling back to dU immediately yields 

X^A ldu = \N \(X, N du )dV du -a{X,N){e, N du )dV du . 
Noting that ae is the tangential component of v then completes the proof. 

■ 

Lemma 5.11. There exists a family 0,$ C E, 5 > sucft £/iai 

• T/ie portion of the boundary dfls in the interior o/E is piecewise C 2 . 

• C(E) C ft«5 /or a// S > 

• -> as (5 -» 0. 

• Jan, IJVoldVki, -» as 5 -» 0. 

w/iere /z™ is i/ie n-dimensional Riemannian spherical Hausdorff measure. 

Proof: By Theorem A.l, the characteristic set C(E) is compact and has Hausdorff 
dimension < n — 1. Thus for any S,p > we can construct a finite collection of 
Riemannian balls U of radius e covering C(E) such that 

J2e n - 1+P < S. 
u 

For each 8 > take fi^ = EnlJ i? for some < p << 1. This family clearly satisfies 

w 

the first three properties. 

The standing assumption on the Riemannian curvature tensor (F) implies that 
for some constant C 



f \N \dVn 5 <e f dVn s +CY^e 

JdQ.s J <9H 



n-1 + 1 



I oris 

which tends to zero as S — > 0. 

■ 

We are now in a position to state and proof the main result of this section, the 
first variation formula for perimeter measure of C 2 surfaces. 

Theorem 5.12. Suppose F is a compactly supported C 1;2 variation o/E with F 
C 2 and variation function p. Then 

d 



dt 



P (St) = / P{div v)dV s - lim / p( v , N Qs )dV n 
*=o+ Js\c(t.) s ^°Jan s 

= / p(div v) — divr.{pv ) dVs 

7e\ 



/E\C(E) 

where Vis is any family satisfying the conditions of Lemma 5.11 and v T is the 
Riemmanian orthogonal projection of v onto TE. 

Proof: Using the existence of a family of neighborhoods of C(E) as in Lemma 5.11 
and pulling back to E, we can immediately decompose 

d 



Po(E t ) = 
t=o+ 



I C dt {F*K) + ± [ (|JV (t)|dK Et ) 

JH\H 5 t =o ^ t=0+ 
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Using the results of [24] as discussed in Theorem 5.5 we can reduce the first term 
to 



L 



p(div v)dVs + / <9 t jF*A 
This equals 

/ p(div v)dVx + I \N \(F^ 
Js\n 6 Jdn 5 at 



t=o 



, N 9 n s )dVon s 

t=o 



p(u T , N d n s }dVan s 



by Lemma 5.10. However the middle term can be neglected as |jV | is Lipschitz 
on S, with the other terms bounded, and so the integral will vanish as 8 — > by 
Lemma 5.11. Thus we need only consider the contribution of 



/ p(div iy)dVj: - p(v T , N a n s )dV 9 n 5 
which by the Riemannian divergence theorem can also be expressed as 

/ p(div v) — divs(p^ T ) dVz. 



Now the second term of (4) decomposes as 

By Lemma 5.9, 1 7V j is Lipschitz on S and vanishes on C(S). The second of inte- 
gral can therefore be uniformly bounded by a fixed constant times 8 Jl £g t LldV^ t . 
Furthermore by Lemma 5.9 again, |AT | has bounded distributional derivative, so 
the first integral is bounded by /z n (fi{). As F is C 1 , J_ Cg t ifdVj: t is bounded on E. 
Therefore as 8 — > the second term of (4) tends to zero. Therefore letting (5^0 
yields the desired result. 

■ 

Corollary 5.13. A necessary and sufficient condition for a C 2 surface to be a 
critical point of horizontal perimeter measure in the category of C 2 hypersurfaces 
with fixed boundary is 

divv = H -^{[^TplTp) =0 

& 

on E\C(S) and 

(5) lim / p{v, Nn 5 )dV n , = - [ div i: (pu T )dVj: = 

S ^°JdQ s JE\C(E) 

for all compactly supported C' 1 functions p. 

Proof: This follows from the fact that every compactly supported C 2 function can 
be realized as the variation function of a C 2 variation of E and our assumption (E) 
that the horizontal mean curvature is in L 1 . m 

Corollary 5.14. A C 2 perimeter critical domain in the category of C 2 domains 
with volume constraint must have boundary S satisfying both div v = c off C(£) 
for some constant c and (5). 
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Proof: Take any open U C M small enough so that dV = dp is exact on U and 
then set U = U fl S. Then any variation with variation function p supported inside 
U must satisfy 

" ' ' ' rp I I) 



dt 

for some constant c. But 
dt 



(p„(E,)-/ a 
/ cp = c£ dt p = / d t jdV + d(d t jp) 



Thus by using a partition of unity we have that 

/ (div v - c) dVz - lim / p{v, N)dV 9 n s 

must vanish for all C 2 functions p on S. Here we can use the same constant c on 
each supporting patch of the partition as the constants must agree on overlaps. 

■ 

Remark 5.15. For any C 2 hypersurface £ such that the characteristic set has 
Hausdorff dimension < n — 1, the family Qg can be chosen so that condition (5) 
is automatically satisfied. This follows easily from the observation that if we follow 
the construction of Lemma 5.11 then dVgn 5 — * 0. For example, as seen in 
[2, 7, 29], in the Heisenberg groups W 11 the characteristic set of any C 2 hypersurface 
has dimension < m. Thus for m > 1 there is no constraint on the characteristic 
set of minimal surfaces. 

Remark 5.16. Let X be a critical point for perimeter variation (with or without 
volume constraint). Suppose that p e C(S) and in a small neighborhood U of p, 
C(S) is an embedded submanifold of dimension n — 1 dividing U into two regions 
U + and U~ . If we further suppose that v extends continuously to v + , v~ on the 
boundaries of U + and U~ respectively, then 

{v+, N c )-{v~, N c )=0 

where N c is the normal to C(S) in U pointing into U~ . 

This follows immediately from the divergence integral form of (5). Note that, 
when restricted to the Heisenberg group, this was observed in [6, 36] where the struc- 
ture of the characteristic locus is known to be either lower dimensional or of this 
form. 

6. Application: a Minkowski formula for CMC surfaces 

Throughout this section we shall suppose M has a globally defined rigid vertical 
complement with decompostion 

TM = V ®($V b 

as in Section 3. 

Definition 6.1. A dilating flow for a subRiemannian manifold with global vertical 
complement is a map D : M x R — > M and constants 7/3, [3 — 1..1 such that 
• (£>a)* maps V to V . 
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• ((D X )*Y, (D x )«Z) DxP = e 2X (Y , Z) p for all sections Y,Z ofV . 

• (£>a)*7> - e^ x T fj for all (3. 

Associated to a dilating flow are the dilation operators defined by 

S\ = AogA 

and the generating vector field X defined by 

X P = ^r D x {p). 

The homogeneous dimension of M is given by 

i 

0=1 

For compactness of notation, we shall write Xp for D\(p) and A*Y for (D\)*Y. 
A dilating flow naturally lifts to a global flow D on the contact bundle So(M). 
If p = (p, v, ap) with [3 = then 

(6) Xp = (Ap, e- x X.«v, eV-T^ap) . 

This lifts ensures that the middle term is still unit length and that if v + apTp is a 
normal vector for the surface E = {<fi = 0}, then e~ x X. t v + eS 1 ~ 1 ^ x apTp is a normal 
vector to Y>\ = {D*_ x (j> = 0}. We also note that tt o D\ = D\ o tt. The generator of 
the lifted flow will be denoted X. 

Lemma 6.2. The contact form p (Y) = ( 7r*Y , v + apTp ) p has the property 

L 2 e = e. 

Proof: We compute 

0\O)p{Y) = 0\p(X*Y) = (tt-AS, e- x X*v + e^-^ x a p Tp ) Xp 
= e A <7r*F, v + a p Tp) p = e x e p {Y). 

The result is a direct consequence. 

■ 

Lemma 6.3. The horizontal perimeter measure form A on Sq(M) has the following 
dilation property: 

£^A = (Q- 1)A. 
Proof: We first note that clearly 

D* x dV = e XQ dV. 
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Now 



(D* x A)p (Y u ..., ?n)= A A p (KYi, . . . KYn] 

= it* (e- x \*vjdV) x$ (\Sl, ■ ■ ■ KYn) 
= e~ x dV\ p (\*v, A*7r*Yi, . . . , A*7r»F„) 
= e-\D x )*dV p (y, nSi, ■ ■ ■ ^S n ) 
= e^-^dVp (y, nSi,...irS n ) 
= e^-^Ap(Y 1 ,...Y n ) 

The result immediately follows 

■ 

In the presence of a dilation, we can define 

T = Q^XjdV 

so that dT = Q^CxdV = dV and £ X T = QT. We also define T = tt*T, 
the pullback of T to So(M). Since D* X T = tt*D* x T, wc immediately see that 
£jjT = QT. 

Now suppose E is a noncharacteristic C 2 hypersurface of M with constant mean 
curvature H. Then E embeds naturally as E into iSo(M) and 

/ C 2 {A-HT) = f (Q - 1)A - QHT. 

But since ir*dV = 9 A A and dA = Hir*dV 

[ C R {A-HT)= [ XjA + f Xj(Hw*dV) — HQY 



XjA. 

Ids 

After pulling back to E along the natural inclusion into So(M), we have now es- 
tablished a Minkowski type identity for C 2 noncharacteristic patches. Namely 

(8) (Q - l)Po(S) = (Q - 1) [ A = QH [ T+ [ XjA 

Jt, jt, Jot, 

Theorem 6.4. Suppose S is a C 2 hypersurface with piecewise C 1 boundary such 
that H is constant off C(E) and E satisfies the constraint (5). TTien 

(Q- l)Po(E) = Q [ HT+ [ XjA. 

Proof: As before we use the family of open sets £1$ containing C(E) constructed 
in Lemma 5.11 and set E5 = E — Clg. 
Then by (8) we know that 

(9) (Q - 1)P (S«) = Q t HT+ [ XjA. 

But by an argument identical to Theorem 5.12, the internal portions of boundary 
integral will tend to zero as S — > 0, leaving the desired equality. 
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Corollary 6.5. Suppose £1 is a compact C 2 domain with S = dfl that is a critical 
point for perimeter measure with volume constraint. Then 

(Q - l)Po(S) = QHVol(Q). 

For the Heisenberg groups, this result was first shown in [36]. 



7. The Second Variation 

We shall now attempt the arduous task of describing a general second variation 
formula under the assumption of rigidity. This unfortunately is just a long tedious 
exercise in computing derivatives using the structural equations of the adapted 
connection in QTv,o{M). The underlying idea is differentiate on both the frame 
bundle and on S x (— e, e) and compare results. 

To aid with the long computations to follow, we shall briefly list the standing 
assumptions and notational conventions of this section. To save time and space, 
we shall also adopt the habit of absorbing all unnecessary terms that do not affect 
the relevant computations into "junk" collections. 

(A) M is a vertically rigid subRiemannian manifold of dimension n+1 = k+l+l. 

(B) Unless otherwise specified £ = is a C°° hypersurface in M with no 
characteristic points. 

(C) F: S x (— e, e) is a C°°'- 3 noncharacteristic variation with F*8 = p and 
Po = Po(',0). 

(D) And recall: roman indices run from 1 . . . k, barred roman indices from . . . k 
and greek indices from 1 . . . I. 

We use the framing dt, F*u>^ , F*r/ a on S x (— e, e) and will generally omit the 
F*. We shall use the notation lu* = uj 1 A • • • A ui k and rf = n 1 A • • • A rf . 

We define a variety of tensors by pulling back the structural equations to S x 
(-e,e). 

F*Jk 



with the understanding that each C 13 , E@ and are skew-symmetric. 

Before diving into the main computation, we shall warm up by using our tech- 
niques to derive an integration by parts formula. 

Definition 7.1. For a differential operator X on S we define the horizontal adjoint 
X* by 

J fXh Ao = J hX*f Ao 

for compactly supported functions /, h. 

The key step to computing the horizontal adjoint of a vector field is the following 
computation on QFvfi'- 



= M a J A rf + B\ (i rf A rf 
= ClJ A J + D?y A if + E^if A rf 
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d(uj^ A rf) = dm® At]* + (-l) k - 1 Jj'> A drj* 
= (-I) 7 "" V A wf A w (mj) A rf 
+ (_l)fc+/J w (i) Af /» Ar? (« 

(10) = (-l) m " V A c^ n A w (m ' J ' ) A ry* + (-l) J ' +m w™ A w (m) A rf 

+ (_l)WM 2C ^o Aw'ArjW 

= (-l)" 1 - 1 ^ A A w( m ' J ') A fy* + (-l) J+m wf A a/ m > A 77* 
+ (-l) fe+/3+J '- 1 2C 3 / Aw'A r/^ + (-l^'-^O/jC^w* A rf. 
Lemma 7.2. For each ej with j > we have 

ef = -ej - 2apC^ - Tf m 
Proof: First note that (10) implies 

(11) A „*)| E = (-lr 1 (2apC^ + ) A . 
Thus 

d(fhu>W Afj*)| S = (-l^'-^/e^ + ^/JAo + a^w^ Ar7*)|s 

Thus 

/" /e,-ftAfl = /" (-/ie,-/ - 2//ia /3 C 3 j - fhTf m )A 

Now we return to the derivation of a second variation formula. We begin 
computing F*d9 = dF*9 in two different ways and equating the results. Firstly 

d9 = d(uj + aprf) 

= uj j A uj° + t° + da f3 A 77 /3 + apf 13 



(12) 



= A lj> + A° Qa uj A ij a + A° ja uj j A V a + dap A rf 
+ a p 2C^u° A J + apD 13 ^ A rf + apD^ ja uP A if 
+ rf A if ■ junk + uj 3 A u m ■ junk 

= -u j A uj 3 + u j A 9( - 2apC^) 

+ uj 3 ' A rf*(A° ja + ( ej a a ) + apD% + 2a a apC%) 
+ 9 A if ■ junk + rf A if ■ junk + uj 3 A uj" 1 • junk 



Thus 



F*d6 = uj 3 A dt( - -yg - 2apC^) 
(13) +w J Ar) a (- rj Q + A° a + ( ej o Q ) + + 2a Q a /J C§) 

+ dt Aii a ■ junk + rj a A ^ • junk + uj 3 A uj m ■ junk 

But from the definitions we see that F*6 = podt so 

d(F*8) = dp A dt = {ejpo)u j A dt + r/ 3 A dt • junk. 
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Comparing with (13) thus yields 

ejPo = -To - 2poa/3Coj 

= -r J 0Q + + (e^) + a p D^ a + 2a a a f3 C% 

Using metric compatibility of the connection thus yields the following useful iden- 
tities 

7o = -ejPo ~ ZpoapC^ 

(15) 

T j 0q = A° ja + a p D^ a + {e a a ) + 2a a apCfc 
Returning to the main computation. Recall that 

# = (-l^VoA^A^ 

and so 

(16) d t jF*y = {-iy- 1 ~tif\uj {j) f\'n*. 

In particular, this implies that 
(17) 

d{d t jF*t>) = {-ly^drf A u>W Ai]* + (-ly-^diuW A rf) 
di&tjF**)^ = (erfo +^Tf m + 2 7 fc /J c£.) A o 

= [eji-ejpo ~ 2p a /3 C 3 .) + {-e-fo - 2p a /3 C 3 .)(rf ro + 2a p C^)] A 
= [ - e^po - AapC^ejpo - Tf m ejp - 2p o e j {a C$ j ) 
- 4p (a^<) 2 - 2ftr£> /J Cg] A . 

Also if we define curvature 2-forms by 

(is) dwUwf a c4+fi;; 

then 

rf* = (-lf'-^^AwWAij*) 

= (-l) j_1 (wj* A < + fij) A w« Ai]* + A d(w« A rf) 

(19) = (-I) 3 '" 1 (wq™ A < + Q J ) A W W) A ry* 

+ (-l) m+ ^ AwjA cj™ A J m - f) Arj* + (-l) m <4 A ojJ 1 A w (m) A rf 



+ (-l) k+0 2C^e Awl A co* A V il3) - 2a CPy o Au>* Arj* 



Thus 



(20) 



-2C^)-2^Cg^la;*A»,* 
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When we restrict to S we can use the fact that H = — T 3 ■ and po = po(-, 0) to see 



T\J ym i tt2 
L Om 1 Oj ' 12 



(21) 



(9 tJ F*rf*)| E = p ((2^ 00j 

- 2apC(j j (-ejpo - 2p o a Cg j ) 

- 2p Cjj j (A% + a 7 D]p + {e jaf3 ) + 2a /3 a a C£,)) A 
= e jPo (2apC^k Q 

+ PO (^^OOj + ^'jrrJ'mj 

~ 2C ojA% ~ 2^^^ - 2Cg(e j a /J ))A 

We now encode all this computation in the following lemma. 

Lemma 7.3. Suppose M is a vertically rigid subRiemannian manifold and F is 
a noncharacteristic C°° :3 variation of S\C(S) with compactly supported horizontal 
variation function. Then 

^L = o^(^)-/ s P t Po)i/A]| t=0 + / e |V°^o| 2 A 

+ jf po 2 [ - ffic v (i/, i/) - Tr(nl) + H 2 

(22) - (Tor&e^^^iToriej,^),!;) 

- {Tor{e h Tp), N v )(Tor(u, e 3 ),T p ) 

- (ej{Tor(y,ej),N v ) - 2(Tor(i/,e j ),N v )(V em e j ,e m )} 

A 



Proof: As was shown Theorem 5.5 

d_ 
It 



So 



4* = °^) 



- {Tor{u,e 3 ),T p ){e 3 ap) - (Tor(v, e 3 ), N v } 2 
5.5 

P (S t ) = fjoF^ 
J^£ dt (p F t *y) | t=Q 

(ftpbjfr + pora,^**) 

(a t po)^A + p ^ t (^**) 



it=o" 



t=0" 



Now as previously shown, locally 
(23) 



t=o 



[E4 e iPo + Po(20 J 00 . - r^r^. + H 2 - 2C^A% - 2a 1 D]pCl ] 



3 = 1 



2e 3 (a p C^) - 2C&(e j a /J ) - 4( a/3 C<£) 2 ) - 21™ a/3 C£. 



A. 



Converting to the invariant form given in the Lemma and integrating by parts once 
then completes the proof. 



24 



ROBERT K. HLADKY AND SCOTT D. PAULS 



Theorem 7.4. Suppose M is a vertically rigid subRiemannian manifold, £ is a 
C 2 hypersurface and F is a noncharacteristic C 1;3 variation of S\C(S). Then 
whenever either of the following holds 

• H = on E\C(S) 

• H is constant on C(S) and F preserves f~ F t */z /or an?/ smooth form with 
H = dV. 



we have 
dt 2 



t=o 



Po(St) = ^ |V°< s Po f A + po 2 [ - ffic v (^, i/) - 7V(F 2 ) 

- (Tor(z/, ej),T )(Tor(ej,Tp), v) 

- ( Tor(e,- ,Tp),N v }( Tor{v, e 3 ) , Tp) 

- (ej(Tor{v,ej),N v ) - 2{Tor(v, ej), N v ) (V em ej, e m )^j 



(Tor(u, ej),Tp)(ejap) - (Tor{v, ej),N v f 



A 



Proof: Since F is supported away from the characteristic set, we have that 
d 



dt 



f F t > = f C dt F^ = jf p (£, *)A t . 



Thus if F preserves volume then J H /5b(£,f)A t = 0. If H is constant then differen- 
tiating yields 



Jjd t p )HA + Po 2 H 2 A Q = 0. 



Therefore the effect of cither condition is that the first term of (22) cancels the 
+po 2 H 2 term within the second integral. Thus the theorem is proved for C°° :3 
variations. 

All that remains is to show that the result still holds with the restricted regularity 
conditions. The difficulty is that for the computations to hold, we must have p 
being C 2 on 5, whereas for a C 1 ' 3 variation we can only guarantee that p is 
continuous. However since F itself is C 2 we see po is C 1 . Fortunately, the right 
hand side of (22) requires only C 1 regularity in p . All the other terms are in fact 
tensorial, so the restricted regularity will not cause problems. 

Now note that 

P** A = A(£, t)d^ A . . . d£ n . 

Furthermore since A is semibasic, we see by Corollary 4.11 that A is C 0;2 . The 
second variation functional 



Po(F(S)) 

t=o 

is therefore continuous from C 1;3 variations to R. By Lemma 4.8 we see that we can 
approximate F by C°°' 3 variations such that the restrictions to t = converge in C 2 
to Fq. The second variation formula of Lemma 7.3 holds for these approximations 
and the formula itself is continuous as a functional on C 2 embeddings. 
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8. Examples 

This second variation formula is hideously complicated in general so we shall 
attempt to illuminate it with some remarks and examples. 

Firstly, recall that the horizontal second fundamental form is asymmetric but 
does have real valued entries. Thus its eigenvalues Ai, . . . A& are either real or come 
in conjugate pairs. From elementary linear algebra we can then deduce 

k 

Trace(iZo) = 

3=1 

^A J 2 = ^Re(A J ) 2 -^Im(A J ) 2 

3=1 3=1 3 = l 

Now the imaginary parts of the eigenvalues reflect the asymmetry of E which in 
turn reflects on the propensity of the tangent horizontal vector fields to bracket 
generate the remaining directions. Reviewing the second variation formula of The- 
orem 7.4 leads to the conclusion that a greater degree of bracket-generating causes 
greater stability in hypersurfaces. 

8.1. Strictly pseudoconvex pseudohermitian manifolds. Recall a pseudoher- 
mitian manifold (M, J, 77) consists of: 

• a 2n + 1-dimcnsional smooth manifold M 

• a non-vanishing 1-form 77 defining the horizontal distribution Vq = ker(r?) 

• a bundle map J : Vq — > Vo such that J 2 = — 1 

with the integrability condition that the Niunhuis tensor (see [40]) vanishes. The 
manifold is strictly pseudoconvex if the Levi metric 

g(X,Y) = d V (X, JY) + r,(X)r,(Y) 

is positive definite. In this instance, a rigid vertical structure can be imposed by 
taking T to be the Reeb vector field of 77, i.e. r/(T) = 1 and T_idrj = 0. 

The second variation formula is then simplified by specializing the adapted con- 
nection to be the Tanaka- Webster connection ([40] ,[41]). Furthermore we require 
the horizontal frame to be J-graded, i.e. Je 2 j = e 2 j+i and J*cj 2j = — w 2j+1 , 
j = 0..n — 1. This implies that 

f = dr, = lu° A lu 1 + u 2 A lu 3 + . . . Lu n - 2 A w"" 1 . 

Thus all the D terms vanish, C 01 = \ and C 0j - = for j > 1. Now 

n-1 

T lrn = E (< V <^ JV > ^ ) + ( V -'^ JV > J ^J )) 
3=1 
n-1 

3=1 
n-1 

3=1 
= (n — l)a. 



H = 

(24) 

Trace(i7 2 ) = 
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we the last line follows from the fact that each [e2j,Je.2j\ must be tangent to S, 
but r]([e2j, J&2j]) = —1 together with the observation that T — av is tangent to S. 

We can now use a combination of Theorem 7.4 and its local expression (23) to 
see 

^ /o(S t ) = J [|V°^ | 2 +Po 2 ( - Ric>,^) - Tr(i7 2 ) 

- (Tor(Jv, T), v) - 2{Jv)a - no 2 ) A 

If the pseudohermitian structure is normal (i.e. VtX = [T, X], see [40] for equiv- 
alent definitions and consequences) then the torsion term vanishes. For the case 
n = 1 this example first appeared in [6], although it should be noted that their 
presentation of pseudohermitian manifolds causes C 01 = 1 instead. 
A few particular examples are especially important in the literature: 



8.2. The Heisenberg Group. H™ = R 2 .™ x R t with the horizontal distribution 
spanned by 

Xj = d xJ - -^y J d t , Yj = d y j + -x J <9 t 

is an example of both a (normal) strictly pseudoconvex pseudohermitian manifold 
(with i] = dt + ^yidxi — ^x^dy^ , JXj = —Yj and Reeb field T = d t ) and a Carnot 
Group. However, the curvature and the horizontal torsion both vanish identically 
so the second variation becomes 

^ P (St)= / f|V ' S po| 2 + Po 2 (-Tr(i7 2 )- 2(J^)a-na 2 )lA 
at |t=o l j 

For n = 1, this example was first shown by Danielli, Garofalo and Nhieu in [12]. 

8.3. The Rototranslation space. Here M — M 2 x S 1 with horizontal distribution 
Vb spanned by 

X\ = cos 9d x + sin 0d y , X 2 = dg. 

A pseudohermitian structure can be imposed upon M, but we shall instead compute 
from first principles. Define 

T = [X lt X 2 ] = sm0d x - cos9d y , 

so [T, Xi] = 0, [T, X 2 ] = —X\. Thus T represents a rigid vertical structure. A flat, 
adapted connection is created by defining 

V T ATi = V T X 2 = 0. 

Direct computation shows that 

Tor(ATi, X 2 ) = -T, Tor(T, X x ) = 0, Tor(T, X 2 ) = X x 
Now for a C 2 hypersurface X we follow [13] by writing the horizontal unit normal 

v = pXi + qX 2 , ei = qXi - pX 2 
We can then compute that 

(Tbr(ei,T),i/) = -f. 

Applying Theorem 7.4, we see that the second variation (of a minimal surface) is 
given by 

^ Po(Z t )= [ \(e lPo ) 2 +p Q 2 ( -p 2 -2e ia -a 2 )]A 
dt 1 |t=o Jt, l j 
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Now if we suppose E is given as the level set of the denning function ip and define 
p = X\ip, q = X2(p, W = vip, u> = Tip and u> = u/W. Then since av — T is tangent 
to E we have a — Co. Furthermore 

ei a=±eiT V >-£e 1 v<p 

= w [ei > T](p -w [euu]ip 

1 / - 
= ^7 {-pTq + qTp - pp) 

- ^ {qvp - pvq + qlXx,^ - p[X 2 ,v]ip) 
= qTp — pTq — p 2 — Co (qvq — pvq) 

- (pqXip + qqXiq + q 2 LO - ppX 2 p - pqX 2 q + p 2 uj) 
= qTp — pTq — p 2 + Co {pvq — qvq) 

- Co (^qXiip 2 + f) - ^pX 2 (p 2 + f)+Co 

= qTp — pTq — p 2 + Co (pvq — qvq) ~ Co 2 
Thus, in this notation, the second variation formula becomes 
d 2 f r 

— P (Et)=/ (eipo) 2 + P0 2 (P 2 + 2{pTq-qTp) + 2Cd(qvq~pvq)+Cd 2 ) 
at* |t=0 J s L 

9. A COMPACT, STABLE CMC SURFACE IN H 2 

In this section, we shall consider in detail the "bubble sets" in the five dimensional 
Heisenberg group H 2 and the connection to the isoperimctric problem. We define 
surface the E C H 2 described as the completion of the double-graph 



L 2 tt L 



2 



t = = ± - - arctan \^=^ ) + - r 2 ) , 0<r<L 

where r 2 = ( x "j + Vj) an( i ^ is a positive constant. This surface is easily 

seen to be C°° away from its characteristic points at (0,±L), and is C 2 but not 
C 3 over the characteristic locus [10]. It is also known to be the only rotationally 
invariant compact CMC surface in H 2 , [37]. In H 1 , this bubble set was shown 
by Ritore and Rosales to be the solution (in the category of C 2 surfaces) to the 
isoperimetric problem, [36] . In addition, Leonardi and Rigot [27] showed the bubble 
set in the first Heisenberg group to be the isoperimetric minimizer in the class of C 2 
rotationally symmetric surfaces. Very recently, Monti and Rickly [31] showed that 
the bubble sets in H 1 are isoperimetric minimizcrs in the class of convex surfaces. 
In all Heisenberg groups, E was shown by Daniclli, Garofalo and Nhieu [10] to be 
the solution to the isoperimetric problem within the category of surfaces that can 
be described as double graphs over discs. 

We shall show that E has constant mean curvature and is stable, in the sense 
that 

i ^o(E t )>0 
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under all volume preserving variations. Whilst not quite sufficient to establish that 
£ is a local minimum, this does at least provide further evidence that S is a viable 
candidate for a solution to the isoperimetric problem in H 2 . 
We start by defining the outward horizontal normal 

where 
and 



■ _ Xj0 y j V L2 - r2 -j _Y^__if_ xWL 2 -r 2 

"^~~W~~~L Lr ' q ~~W~T + Lr ' 

Then Jv = q J Xj — p>Yj. We extend v, Jv to a horizontal frame by first defining 

e = - r (x 2 X 1 - x 1 X 1 - y 2 Y x + y x Y 2 ) 

J e = \ {v lx 2 - V 2 Xx + x x Y 2 - x 2 Y x ) . 
Now a straightforward, if brutal, computation shows that 





i Vi 



L Lr 
VL 2 -r 2 1 
Lr L 



Thus the eigenvalues are 



k = 



2 
I 

i .Vl 



Ki,K2 = — ±i 



Lr 



So we can compute 



H 



L 

traced) = A__|_ (L2 _ r2) 



Now 



Lr 

thus a simple computation shows 

2 

Jv(a) = 



Hence 



2 6 4 L 2 -r 2 



trace (H 2 ) - 2Jv{a) - 2a 2 = -^(L 2 - r 2 ) - + 



L 2 r 2K ' L 2 r 2 L 2 r 2 



So 

(25) ^ |(=/0(S , ) = / c (|v^| 2 -^|A 
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at least for variations supported away from the characteristic locus. However, direct 
computation shows that 

\N - ' 



- 4r 2 + L 2 r 2 

and so |AT | is comparible to r near r = 0. For a variation supported over the charac- 
teristic set, the horizontal variation function can then be expressed as po = r^r and 
the integrand of right hand side of (25) is again integrable. Since the characteristic 
locus is zero dimensional, the first variation formula holds without a boundary term 
even for variations supported over the characteristic locus. Reviewing the proof of 
the second variation formula reveals that the only place that required support away 
from the characteristic locus was the final integration by parts. If however, we cut 
out the characteristic locus by shrinking discs of radius e and apply Lemma 7.2, 
we see that the boundary terms take the form po^°'^Po multiplied by a differential 
form bounded by the size of A. The boundary integrand blows up at a rate bounded 
by = ^ • e. Due to the small size of the characteristic set, the boundaries of 
these discs are shrinking at rate e 3 and so the boundary term is also negligible for 
characteristic variations. Hence (25) holds for all variations. 

To aid calculation, we now switch to polar coordinates po = po (r, a) so that 



dV = sf\ + W 2 r 3 drda 

and 

A = - = dV = XMdrda 

Vi + w 2 y ' 

where 

A(r) = 



VL 2 - r 2 

We note that away from the characteristic points, the level sets of the radial 
function r foliate E by 3-dimensional spheres. Now L 2 (S 3 ,C) can be orthogonally 
decomposed into (complex) homogeneous polynomials of bidegrce p, q. Therefore 
on each foliating sphere we can decompose po into spherical harmonics in L 2 (S 3 ), 

Po(r,<r) = Pa 9 ( r ) (J P,i- 

p,q>Q 

Remark 9.1. There is a natural pseudohermitian structure on the odd dimensional 
spheres and many natural subRiemannian operators have been studied using this or- 
thogonal decomposition, see for example [19], [23]. This technique can be generalized 
to study foliations by other compact pseudohermitian manifolds, [40] , [22] . 

If we introduce the complex coordinates = + iyi , we can express Jv as, 



Jv = ^L 2 - r 2 d r + i (fdgj - z j d zi ) 

and so 

Jv{po)= (fir + iil - P)\ P™) a P,q 

P,9>0 

Now if Z — z 2 d z i — z 1 9 z 2 then 

e = Z + Z, Je = Z - ~Z. 
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/ 

Js 3 



Now as a differential operator on L 2 (§ 3 ), Z* = — Z so 
\e(Po(r, cr))| 2 + \Je (p a (r, a))\ 2 da 

= ~2 [ ~Po(Z + Z ){Z + Z)p - p (Z - Z)(Z - z)poda 
r Js 3 

= i / -2p {ZZ + ZZ)p da 
r JS 3 



r 
p,q 



Thus 



71—1 

/ e(i^i 2 +i^poi 2 )a> / 4( P0 - P r) 2 A 

m=l ^ r 

By spherical orthogonality, stability is proved if we can show 

Using the transformation r = Lsin0, < 9 < ir, this reduces to showing 
£ 2L 2 (d e p° ) 2 sin 4 9d0 > £ AL 2 ' sin 2 0d0 

Now Pq itself is generically singular at 9 — 0, 7r and indeed Pq'° may not be in 
L 2 (0,7r). But h — pg^sin^ is necessarily continuous up to 9 = 0,ir. It is also easy 
to see that dgh must be continuous and L 2 on (0,7r). 

Recall that we are only interested in variations that preserve the volume, i.e. 
J E poA = 0. With these transformations this implies that /J r /isin 3 9 d9 = 0. 

Thus stability for the geodesic ball in I" is equivalent to the statement: for all 
h G C[0,tt] flC^O.f) such that hi e i 2 (0,7r) and 



/isin 3 6»^ = 

o 



the inequality 

/■7T /"7T 

" dff 



I {hi sm9- h cos 9f d9> / 2h 2 
Jo Jo 



holds. 

The proof of this inequality is an elementary computation using a Fourier de- 
composition and is provided in the appendix. 

Remark 9.2. It seems reasonable to conjecture that this result would also apply 
with n > 2. However the required computations are more complicated due to the 
fact that for n > 2, there is no useful global horizontal frame, even away from 
characteristic points. This is essentially the classical result that higher dimensional 
spheres are not parallelizable. 
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Appendix A. Size of the characteristic set 

In this section we prove 

Theorem A.l. Suppose Xi,X 2 ,---Xi c are smooth globally defined vector fields 
on E™ that bracket generate at every point. Then for any C 2 hypersurface E, the 
characteristic set 



C(E) = {peE: (Xj) 
has Hausdorff dimension < n — 2. 



£ T p E for all l<j< k} 



The size and nature of the characteristic locus has been studied widely [2, 16, 28] 
in various contexts. We include a discussion here for completeness and because, to 
the best of our knowledge, a complete argument for general sub-Riemannian spaces 
does not appear in the current literature. 

The proof is based on a series of technical lemmas. 

Lemma A. 2. Suppose f is a C 1 function on K m . Then the set 



V = {p: f(p) = 0, df 



^0} 

p 



has Hausdorff dimension < m — 1 . 

Proof: Fix p e V. Then since the set K = {p: f(p) = 0, df\ p — 0} is closed, 
we can find an open set U containing p such that f\jj is a C 1 submersion from 
U into K. The constant rank theorem implies that U n V is a closed embedded 
submanifold of U. The set U n V thus has Hausdorff dimension to — 1 as a subset 
of U (and hence as a subset of R m .) 

Therefore we can cover V by open sets U a such that each V n U a has dimension 
< m — 1. Since every subset of R m is second countable we can find a countable 
subcover by the Lindelof theorem. Thus we can express V as a countable union of 
sets of dimension < m — 1, which is sufficient to prove the result. 

■ 

Our next lemma is a refinement of a result due to Derridj, Lemma 1 in [16]. 

Lemma A. 3. Suppose E is a C 2 hypersurface in R" and X and Y are smooth 
vector fields. Then the set 

V = {p e E : X p , Y p e T p E, [X, Y] p <£ T p E} 

has Hausdorff dimension < n — 2. 

Proof: Locally we can introduce C 2 slice coordinates (y, x 1 , . . . , a;™" 1 ) so that 
E = {y = 0}. Rewrite X and Y in these coordinates as 

X = ad v + a l d x i 



Y = bd y + b l d x 



Then 



„, f db da . db ,.• da \ „ , „ „ 

l*> n - {'oj -% + al 9 x - i - bl ^) Q y mod • • • d * 
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The condition that p = (0,p r ) £ V is therefore equivalent to 

o(p) = 
< b(p) = 

Set a' = 0|s, = &|e- The portion of V lying inside the slice coordinate chart must 
be contained in {a' — 0, da' ^ 0} U {&' = 0, db' ^ 0}. Since a' and 6' are (at least) 
C 1 functions on R™" 1 , the result now follows from Lemma A. 2. 

■ 

Proof of Theorem A.l: Generate the countable collection of all vector fields 
that can be bracket generated by X\ , . . . X k and enumerate them as 

X\ , . . . Xk , • • • , X a , . . . 

with the first k matching the original vector fields. Define 

E a fi = {peS: (X a )\ p , {Xp)\ p e TpYi, [X a ,Xp] p T p £}. 

Thus {E a p} is a countable collection of sets of Hausdorff dimension < n — 2. But 
since the original vector fields bracket generate at every point, for every p € C(S) 
we must be able to find X a and Xp such that p £ E a p. Therefore C(S) is contained 
in the countable union of sets of Hausdorff dimension < n — 2 and so must have 
dimension < n — 2 also. 

■ 

Remark A. 4. Without further restrictions on the vector field Xj this result is 
sharp for all n > k>2. To see this set 

Xj=d xj forl<j<k-l 

x k = d x u + { x l )d xk+ i + {x l fd xk+ 2 + ■■■ + (x 1 )"-^^. 

These vector fields bracket generate at step n — k + 1 at all points ofR n . The smooth 
surface S = {x 11 = (a; 1 ) 2 } then has the property 

{s n = .T 1 =0}cC(S). 

Thus the Hausdorff dimension of C(E) must be > n — 2. 

In the special case of the higher Heisenberg groups this theorem is decidedly non- 
sharp. It was shown by Balogh, [2], that for the Heisenberg group of dimension 
2n + 1 the characteristic set dimension is bounded by n rather than 2n — 1. Balogh 
also showed that if the condition C 2 is relaxed to C 1,1 then the bound < 2n is 
actually sharp. 

The improved bounds for the higher Heisenberg groups was independently shown 
by Cheng-Hwang [7] for graphs over the horizontal variables. Their technique had 
the advantage that it used only elementary linear algebra and generalized to graphs 
in pseudohermitian manifolds in natural coordinates. Here we present a new coordi- 
nate free version of this approach which can be used as a tool to study characteristic 
dimension in general cquircgular subRicmannian structures. 

Definition A. 5. Given a collection of vector fields X = {Xi, . . . X n } and a C 2 

function <f>, we define the Hessian of <f> at p with respect to X by 

X\^p) = {X 3 X k ^p) 
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Additionally we define the symmetric Hessian and skew- symmetric Hessian by 

xl = x 2 + (x 2 ) T , xl = x 2 -(x 2 ) T . 

Thus we note 

(26) X 2 _{<j>,p) = ([X j ,X k \ p <j>). 

Now at any point p, we can find a non-degenerate constant matrix P such that 
the skew-symmetric Hessian can be written 

'J\ 



X =p- 







where J\ is the 2A x 2A matrix with A copies of y ^ along the leading diagonal 

and zeros everywhere else. Thus we immediately obtain that 

rank (X 2 ) = 2A, rank (X 2 ) > A. 

As a basic illustration of the use of these Hessian we present the following lemma, 
which essentially first appeared in [7]. 

Lemma A. 6. Suppose (M, n, J) is a 2m+l dimensional pseudohermitian manifold 
such that the Levi form 

(X, Y) i ► dn(X, JY) = -n[X, JY], X, Y horizontal 

has signature (p,n) withp + n > 2k everywhere. Then is £ is any C 2 hypersurface, 
the characteristic set C(£) has Hausdorff dimension < 2m — k. 

Proof: Choose p E C(E) and let <fi be a C 2 defining function for £ in a neigh- 
bourhood of p. Next choose X = {X\, . . . AT 2m } with X m+ j = JX rn a frame for the 
horizontal distribution near p. Now T(f> cannot vanish at p as otherwise d<f>i p = 0. 
Since the Levi form has total signature bounded below by 2k, from (26) have that 
Tank(X 2 _((j),p)) > 2k. Thus rank (X 2 ((j),p)) > k. 
Define a function F: M -» R 2m+1 by 

X 

If we extend X to by a vector field T to a frame for M near p we see 

DF v=(x 2 {^p) T t 

thus DF p has rank > k+ 1. But near p, C(£) = F _1 (0) so the intersection of C(S) 
with a neighbourhood of p is contained in an embedded submanifold of dimension 
<2m-L 

■ 

This technique can be extended to equiregular subRicmannian manifolds of 
higher step or otherwise more complicated vertical structures, but the generically 
the derived bounds on characteristic dimension are no better than the general result 
of Theorem A.l. 

Suppose M is an n-dimensional equiregular subRicmannian manifold M with X 
a smooth local frame for the horizontal distribution. Then we can produce smooth 
frames X^,X^ 2 ), ■ ■ ■ consisting of vector fields produced from X by 1,2, . . . or less 
commutations respectibely. Then for any C 2 surface S and any point p e C(£) we 



34 



ROBERT K. HLADKY AND SCOTT D. PAULS 



again study the skew-symmetric Hessian X 2 (</>, p) . If this Hessian does not vanish 
at p we can deduce that C(S) is locally contained in an embedded submanifold 
of dimension < n — 2. If the X 2 does vanish we can immediately deduce that p 
is actually a characteristic point for the distribution Xm- We then iterate this 
argument. If X is bracket-generating, this must terminate and we have rederived 
the result of Theorem A.l. 

If the step size of the subRiemannian structure is greater than 1 , then in partic- 
ular this argument suggests that gcncrically we cannot expect any improvement on 
the bound n — 2. This would not be surprising as the condition that the dimension 
of the the hypersurface equaling the dimension of the horizontal distribution might 
be expected to yield a richer theory than the general case. That said, there are 
examples where this technique can produce improved bounds. 

Example A. 7. Consider M = H H the 21-dimensional manifold constructed as fol- 
lows: 

Xj. k = d X] k - -Vj,kUj, Yj. k = d V] k + -Xj.kUj, j, k = 1..4 

with the Uj the horizontal generators of an independent copy of H 2 . Thus M 
consists of 4 copies of H 2 each yielding an clement of another H 2 as its characteristic 
field. M is then a step 2 Carnot group with codimension 5 horizontal distribution. 

If £ is a C 2 hypersurface with defining function </> then for any p e C(S), either 
X-{4>iP) vanishes identically or has rank > 4. However if A" 2 vanishes identically 
then p is a characteristic point for X^iy But this higher level characteristic set is 
contained in a submanifold of dimension < 21 — 3 by an identical argument. □ 

Appendix B. Technical Lemma 
This section is devoted to proving the following technical lemma. 
Lemma B.l. Suppose that p e C[0,tt] n C^ . 71 ") and p' <= L 2 (0,ir). If 

psin 3 ede = o 

Jo 
then 

/ (p' sine - p cose) 2 de > 2 / P 2 de. 

Jo Jo 
Proof: We first note that 



/" 

Jo 



(p' sine — pease) 2 = ([psinfl]') 2 — App' sin 9 cos 9 
Now by an integration by parts 

/ App' sin 9 cos 9d9 = - 2p 2 (cos 2 9 - sin 2 9) d9 
Jo Jo 

= - 2p 2 d9 + A / (psin9) 2 d9 
Jo Jo 

Thus 

(27) / (p'sin9- pcos9) 2 -2p 2 d9 = / ([psine]') 2 - A (p sin 9) 2 d9. 
Jo Jo 
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The lemma is proved if we can prove that this integral is positive. We proceed 
by splitting psin9 into its Fourier decomposition on (0, 7r), i.e. as an element of 

psin 6 = y + K cos(2fc6>) + b k sin(2fc0)) . 

fe>i 

Therefore the integrals in (27) equal 

(28) |(-^ + E( 4fc2 - 4 )(^ + ^) 2 

Now the problem is the constant term a , however we have yet to use the fact that 

/ (psm9)sm 2 0d0 = O. 
Jo 

Writing sin 2 9 = — sin(20)) we see that ao = 02- Thus using the term k = 2 we 
see 

-f + (16-4)(a 2 + 6 2 ) 2 >0. 
The integrals of (27) are therefore positive and the lemma is proved. 

■ 

Corollary B.2. The only function p satisfying the conditions of Lemma B.l such 
that the inequality is an equality is p = a\ cosd. 

Proof: For k 7^ 1, the inequality at each level of the sum in (28) is strict. Thus p 
must have a k = W = for k ^ 0. But if 61 7^ then psin# is not orthogonal to 
sin 2 6. 
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